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L  INTRODUCTION 


Over  the  past  several  years,  the  computational  power  of  mainframes  and  personal 
computers  has  increased  manyfold.  Much  larger  and  more  comply':  optimization 
models  (involving  multiple  scenarios)  can  be  solved  in  less  time.  This  implies  a 
commensurate  growth  in  the  size  of  output.  The  sheer  volume  of  output  from  these 
models  is  overwhelming;  multiple-run  comparisons  involve  even  more  output.  Current 
analysis  techniques  have  become  awkward  and  time  consvuning. 

Given  suifrcient  time,  detailed  output  analysis  for  almost  any  model  can  be 
accomplished.  However,  decision  makers  usually  place  time  restrictions  upon  the 
analyst.  The  scope  of  analysis  usually  must  be  narrowed  down  to  some  manageable 
size.  The  application  of  automated  analysis  tools  offers  the  potential  to  do  more 
thorough  analysis  in  less  time. 

Computer  tools  are  necessary  to  automate  the  analysis  process  and  manage  the 
vast  amount  of  data  and  output.  The  aim  of  these  tools  is  to  enhance  the  analysis 
process  by  improving  the  quality,  speed  and  scope  of  output  analysis,  and  reduce  data 
input  errors.  The  design  of  visualization  tools  to  aid  operations  research  analysis  must 
be  preceded  by  a  carefril  evaluation  of  the  decision-making  process.  Applicable 
visualization  principles  [Ref  l:p.  11-12]  must  be  incorporated  into  the  design.  There 
are  many  different  ways  to  present  information,  therefore,  design  choices  must  be 
made. 
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The  focus  of  this  thesis  is  to  (1)  analyze  a  decision-making  process  and  identify 
analysis  choke  points,  (2)  design  analysis  tools,  and  (3)  demonstrate  them. 

An  important  operations  research  model,  HEAVY  ATTACK,  has  been  selected 
to  demonstrate  these  tools.  This  large-scale  optimization  model  has  been  used  by  the 
Combat  Forces  Division  at  Headquarters,  United  States  Air  Force,  for  over  two  decades 
to  generate  air-to-ground  munitions  requirements  for  the  Air  Force  Statement  of 
Reqvurement  in  order  to  justify  procurement  objectives.  HEAVY  ATTACK  is  widely 
accepted  by  the  Department  of  Defense  and  the  Congress,  and  is  used  to  help  plan  the 
expenditure  of  over  $2  billion  each  year.  Recently,  HEAVY  ATTACK  was  used  by  the 
Combat  Forces  Division  for  analysis  of  munitions  requirements  for  Operation  Desert 
Storm. 

Chapter  n  provides  a  general  view  of  the  HEAVY  ATTACK  model.  Chapter  HI 
describes  the  ctirrent  difficulties  with  using  HEAVY  ATTACK  and  with  anal^’zing  its 
output.  Steps  to  improve  the  analysis  are  described.  Chapter  IV  discusses  the 
implementation  of  the  analysis  tools.  These  tools  are  based  on  spreadsheets.  Chapter 
V  states  the  conclusions. 
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n.  HEAVY  ATTACK  MODEL 


«  For  over  25  years  mathematical  programming  has  been  the  foundation  for  the 

budget  planning  of  air-to-groimd  conventional  ordnance  by  the  Combat  Forces  Division 

» 

within  the  Directorate  of  Operations  of  the  United  States  Air  Force.  The  planning 
goal  is  to  ensm-e  sufficient  quantities  of  war  reserve  munitions  are  purchased  annually. 
The  Division’s  Air-to-Ground  Nonnuclear  Consumables  Annual  Analysis  (A/G  NCAA) 
(Figure  2-1)  currently  uses  three  programs  —  SELECTOR,  HEAVY  ATTACK,  and 
HEIAVY  GOAL  —  to  accomplish  this  goal.  These  models  reside  on  a  mainframe 
computer  in  the  Pentagon. 

A.  A/G  NCAA  METHODOLOGY 

In  the  annual  analysis  process  the  fu-st  model,  SELECTOR,  determines  weapons 
effectiveness  based  on  the  Joint  Munitions  Effectiveness  Manual  (JMEM)  [e.g.  Ref  2], 
weather,  aircraft  attrition  and  cost.  The  JMEM  lists  the  average  number  of  targets 
of  type  j  killed  by  sortie  type  i  using  a  specified  tactic  t  in  a  given  weather  of  type  w. 
SELECTOR  takes  the  JMEM  data  and  produces,  as  an  input  to  HEAVY  ATTACK,  the 
average  number  of  targets  of  type  j  killed  by  sortie  type  i  [Ref  3:p.  4].  The  data  is 
then  sorted  from  the  most  cost-effective  to  the  least.  Because  the  most  effective 
weapon  is  usually  the  most  expensive  or  because  there  may  be  a  high  attrition  rate  to 
^  the  aircraft  when  employing  a  specific  tactic,  SELECTOR  chooses  the  most  cost- 

effective  tactic. 
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Weapon  Effectiveness  Data  Base 

Attrition  Data  Base 

Cost 

Weather 


War  Mobilization  Plans 
Target  Data  Base 
Theater  Cdr  s  Assumptions 


G'-ouno/Sea  Lane  Attrition 
Expend  ture  Factors  for 
C  aff.  Flare,  others 


Figure  2-1.  A/G  NCAA  Methodology  Overview. 

The  second  program,  HEIAVY  ATTACK,  is  the  cornerstone  of  the  planning  effort. 
It  is  a  FORTRAN-hased,  large-scale  optimization  program  with  a  nonlinear  objective 
function  and  linear  constraints.  For  each  of  seven  time  periods,  varying  in  duration 
from  hours  to  months,  HEAVY  ATTACK  uses  the  SELECTOR-nominated  optimum 
weapon  for  each  aircrafl/target  combination,  loads  that  weapon  onto  the  aircraft,  and 
allocates  aircraft  sorties  to  each  target.  The  model’s  objective  is  to  maximize  the 
expected  value  of  targets  destroyed  taking  into  account,  among  other  things,  aircraft 
attrition,  the  ability  of  targets  to  regenerate  themselves,  weather,  and  Von 
Clausewitz’s  "fog  of  war"  [Ref  4}.  In  essence,  HEAVY  ATTACK  fights  a  multi-period, 
theater-level  war  to  determine  the  requirements  for  ordnance  stock. 
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The  Combat  Forces  Division  operates  these  models  and,  for  each  respective 
m^or  Air  Force  theater,  hosts  an  individual  annual  week-long  air-to-ground  NCAA 
planning  conference  at  the  Pentagon.  Senior  staff  ofRcers  bring  their  commander’s 
4  theater  plan  for  their  conference.  This  HEAVY  ATTACK  input  is  what  the 

commander  assesses  is  his  mission,  the  threat  he  is  facing,  the  enemy’s  order  of  battle, 
and  his  war  fighting  strategy.  Other  data  input  requirements  for  HEAVY  ATTACK 
include  the  sortie  effectiveness  data,  weather,  aircraft  availabUity  and  weapon 
inventory.  Aircraft  availability,  provided  by  the  Combat  Forces  Division,  is  obtained 
from  the  Air  Force  War  and  Mobilization  Plan,  which  allocates  tactical  air  forces 
between  the  theaters.  Weapon  inventories  are  apportioned  among  the  theaters  by  the 
Combat  Forces  Division.  [Ref.  5]  HEIAVY  ATTACK  takes  these  inputs  and  produces 
the  weapon  reqxiirements  for  each  theater. 

When  each  theater  has  completed  its  annual  visit,  their  requirements  are 
combined  to  form  an  input  to  the  last  model  —  HEAVY  GOAL.  HEAVY  GOAL  takes 
the  combined  requirement,  plus  attrition  rates  for  sea  lift  and  in-theater  ground 
transportation,  and  furnishes  the  aggregate  weapon  requirements  for  the  Air  Force 
Program  Objective  Memorandum  (POM). 

B.  HEAVY  ATTACK  OPTIMIZER 

In  1974  the  Air  Force  adopted  a  nonlinear  programming  model  written  by  Clasen, 
Graves,  and  Lu  from  RAND  Corporation  to  "select  an  air-to-surface  munitions  mix  to 
.  be  stockpiled  as  war  reserve  inventory."  [Ref  6:p.  1]  The  objective  was  to  maximize 

the  "military  worth"  of  targets  destroyed  by  allocating  sorties  to  targets.  Their  single 
period  mimitions  mix  methodology  consisted  of  three  steps:  (1)  selecting  a  munition 
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for  each  aircraft-target  combination  based  on  a  criterion  of  least-cost-per-expected- 
target-killed;  (2)  allocating  aircraft  sorties  to  targets,  with  each  sortie  carrying  the 
selected  "optimum*  munition;  and  (3)  computing  the  required  munitions  miit  Figure 
2-2  shows  the  relationship  between  these  steps  and  various  inputs. 


C.  INPUTS  TO  HEAVY  ATTACK 

There  are  many  sources  of  data  for  HEAVY  ATTACK  for  each  theater.  The 
commander’s  theater  plan  provided  to  the  Combat  Forces  Division  is  one  mqjor  input; 
it  is  in  the  form  of  the  Target  Data  File  described  below. 

Another  input,  the  weather  distribution,  plays  an  important  role  in  how  HEAVY 
ATTACK  allocates  aircraft  to  targets.  The  distribution  forecasts  meteorological 
conditions  for  the  theater  in  the  form  of  prior  probabilities  for  six  discrete  weather 
bands  ranging  from  foul  to  clear  weather. 


6 


Aircraft  availability  and  weapons  inventoiy  constitute  the  remaining  nugor  inputs 
and  are  provided  to  the  theater  teams  hy  the  Combat  Forces  Division. 

Nine  input  files  are  required  for  HEAVY  ATTACK.  These  files  are  classified  as 
4  data,  exclusion,  or  other.  Currently,  all  files  reside  on  a  Pentagon  mainframe 

^  computer.  They  are  modified  using  the  mainframe’s  text  editor.  The  data  and 

exclusion  files,  which  represent  the  nugority  of  the  data,  are  described  in  detail  below. 
Since  the  "other"  files  are  easily  updated  with  the  mainframe  editor,  they  are  neither 
included  in  the  analysis,  nor  described. 

1.  Data  Files 


Targets,  aircraft  and  weapons  are  defined  in  base  data  files. 
a.  Target  Data  File 


The  target  file  (Figure  2-3)  is  the  conunander’s  theater  plan.  There 
have  been  as  many  as  85  different  target  types. 


1992  NFS,  UNCLASSIFIED 

TARGET  values 

REGENER 

TGT 

QTY 

TE 

VI 

V2 

V3 

V4 

V5 

V6 

V7 

CC  DAYS  QTY  F/M 

1(00000)TG01 

16. 

2 

14.0 

12.0 

8.0 

8.0 

8.0 

8.0 

8.0 

.20 

5. 

17. 

TARGET 

01 

2(00000)TG02 

8. 

1 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

10.0 

.50 

2. 

8. 

TARGET 

02 

3(00000)TG03 

324. 

1 

1.0 

5.0 

15.0 

15.0 

15.0 

15.0 

12.0 

.50 

2. 

16. 

TARGET 

03 

<i(00000)TG0« 

162. 

1 

1.0 

5.0 

15.0 

15.0 

15.0 

15.0 

13.0 

.50 

2. 

16. 

TARGET 

04 

5(00000)TG05 

10. 

1 

5.0 

5.0 

10.0 

10.0 

10.0 

10.0 

10.0 

.20 

10. 

0. 

TARGET 

05 

• 

• 

• 

• 

• 

• 

B5(00000)TG85 

1180. 

3 

2.0 

2.0 

1.0 

1.0 

1.0 

1.0 

1.0 

.30 

2. 

28. 

TARGET 

85 

Figure  2-3.  Example  of  an  unclassified  HEIAVY  ATTACK  Target  Data  File. 
The  United  States  Air  Force  provided  these  numbers  from  an  unclassified  data 
set.  The  heavy  dots  indicate  a  continuation  of  data. 


(1)  Target  Name  and  Description.  The  name  may  be  up  to  four 
characters  and  the  description  up  to  15.  Two  examples  are  ’ABHU  —  AC  Bkr, 
Hrd,Ugnd’  (Hardened,  Underground  Aircraft  Bunker)  and  ’SA9  —  SA-9  SAM  battery’ 
(a  Soviet-made  siuface-to-air  missile  (SAM)  battery). 
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(2)  Quantity.  The  number  of  targets  within  each  target  type. 

(3)  Target  Element.  The  number  of  elements  of  a  target  type.  For 

example,  each  of  the  40  tank  parks  (quantity  of  targets)  may  contain  five  tanks  (target 
elements).  » 

(4)  Time  Weighted  Target  Value.  A  value  given  to  each  target  type  ^ 

for  each  of  the  seven  time  periods.  This  value  indicates  how  important  it  is  to  the 
commander  to  kill  a  target  type  relative  to  all  other  types  for  a  given  time  period.  For 

example,  a  SAM  battery  would  be  weighted  high  in  the  first  two  to  three  periods  and 
low  in  the  remaining.  Because  SAM’s  are  a  very  high  threat  to  aircraft,  it  makes  sense 
to  destroy  them  at  the  beginning  of  the  war  in  order  to  achieve  air  superiority,  hence 
the  high  initial  value.  On  the  other  hand,  destroying  a  logistics  target  at  the 
beginning  of  the  war  does  not  make  military  sense;  therefore,  they  have  a  low  initial 
value  and  are  higher  in  the  last  few  time  periods. 

(5)  Coefficient  of  Confirmability.  This  number,  between  zero  and  one, 
symbolizes  the  ability  of  a  pilot  to  assess  his  battle  damage  to  the  target.  The 
coefficient  mathematically  expresses  Clausewitz’s  "fog  of  war"  principle.  If  pilots 
caimot  confirm  kills,  or  discriminate  between  live  targets  and  destroyed  targets,  they 
will  waste  attacks  and  mrmitions  on  already  destroyed  targets.  The  number  one 
indicates  a  pilot  is  unable  to  confirm  the  effect  of  his  first  weapons  delivery,  while  zero 
represents  perfect  target  damage  assessment  [Ref.  7:p.  14]. 

(6)  Regeneration  Days  and  Quantity.  These  specify  the  average 

4 

number  of  days  required  to  regenerate  a  destroyed  target  type  and  the  number  of 
replacements  the  enemy  has.  Regeneration  is  the  process  by  which  destroyed  targets  ' 

are  replaced  with  available  spares  (a  pontoon  bridge  for  a  destroyed  spanning  bridge) 
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or  repaired  (filling  in  the  bomb  craters  on  a  runway).  Additional  information  may  be 
fo\md  in  [Ref.  8:p.  14]. 

b.  Aircraft  Data  Pile 

This  file  (Figure  2-4)  contains  the  aircraft  allocated  to  the  theater  by 
the  Air  Force  War  and  Mobilization  Plan.  There  have  been  as  many  as  25  different 
t3^es  of  aircraft. 
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• 

• 
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3 
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0 

3 
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0.92 

0.00 

• 
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• 
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13 

AC  IB 

222 

305 

252 

437 

342 

318 
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1 

0.95 

0.95 

0.95 

0.95 

0.95 
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0.00 

Figure  2-4.  Example  of  an  unclassified  HEAVY  ATTACK  Aircraft  Data  File. 


(1)  Selector  Number.  The  SELECTOR  file  index  of  this  aircraft  type. 

(2)  Weather  Number.  The  number  one  indicates  the  aircraft  is  all- 
weather  radar  delivery  capable. 

(3)  Aircraft  Name.  Up  to  four  characters,  e.g.  F15C,  AlOA 

(4)  Category  Label.  Each  aircraft  type  is  placed  into  one  of  nine 

•  available  categories  for  the  output  reports.  Examples  are  FALCON  and  EAGLE.  This 

aggregation  is  used  by  HEAVY  GOAL  to  identify  collateral  support  requirements. 
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(5)  Aircraft  Category  Number.  This  identifies  the  category  of  each 


aircraft. 


(6)  Number  of  Sorties.  Identifies  the  total  number  of  sorties  each 

aircraft  type  is  capable  of  flying  for  each  time  period.  • 

(7)  Attack  Mission  Percentage.  The  percentage  of  sorties  devoted  ^ 

solely  to  air-to-ground  attack  missions  for  each  time  period. 

c  Wecpon  Data  File 

All  theaters  use  the  same  weapon  data  base;  there  have  been  as  many 


as  65  weapon  types  (See  Figure  2-5).  For  planning  purposes  the  allocation  of  weapons 
among  the  theaters  is  decided  by  the  Combat  Forces  Division. 


WPN# 

IDEHT 

DESCRIPTION 

P/NP 

WPN  COST 

MAX  EXPEND 

1 

WPOl 

WEAPON  1 

0 

0 

7710. 

2 

WF02 

WEAPON  2 

0 

0 

547. 

3 

WP03 

WEAPON  3 

0 

0 

412B4. 

h 

HF04 

WEAPON  4 

0 

0 

0. 

5 

WP05 

WEAPON  5 

0 

0 

5002. 

# 

• 

• 

• 

• 

• 

65 

WP65 

WEAPON  65 

0 

0 

270. 

Figure  2-5.  Example  of  an  imclassified  HEAVY  ATTACK  Weapon  Data  File. 


(1)  Weapon  Number.  A  numeric  index. 

(2)  Weapon  Name  and  Description.  The  name  may  be  up  to  four 
characters  and  the  description  up  to  25.  An  example  is  LGB  —  Laser  Guided  Bomb. 

(3)  Play/No  Play.  This  is  a  binaiy  switch  (zero,  one).  The  number  one 
removes  the  weapon  from  HEAVY  ATTACK 

(4)  Maximiun  Expenditure.  States  the  inventory  allocated  to  the 


theater. 
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2.  Exclusion  Files 


Not  eveiy  aircraft/target/weapon  combination  is  desirable  or  feasible  in  a 
sortie  plan.  Some  convenient  method  is  necessary  to  exclude  particular  combinations. 
Of  the  possible  combinations,  HEAVY  ATTACK  accepts  inadmissable  lists  of  aircraft 
versus  targets,  aircraft  versiis  weapons,  and  aircraft/targets  versus  weapons.  These 
exclusion  files  are  used  to  preclude  combinations  which  are  not  militarily  feasible. 

The  data  in  each  exclusion  file  is  in  pairs  of  rows.  For  example,  in  the 
Aircraft  versus  Target  Exclusion  File  shown  in  Figure  2-6,  the  first  row  in  each  pair 
is  the  aircraft  t3rpe.  The  following  row  contains  the  target  t3rpes  which  will  never  be 
attacked  by  its  paired  aircraft  type. 


1 

1  2  3  4  S  6  7  8  9  10  11  12  13  14  IS  16  17  18  19  20  21  22  23  24  2S 

1 

26  27  28  29  38  40  41  45  47  51  56  57  59  60  61  62  63  64  66  67  69  71  72  73  74 
1 

75  76  77  78  79  81  85 
2 

1  2  3  4  5  6  7  8  10  11  12  13  14  IS  16  17  18  19  20  21  22  23  24  25  26 

• 

• 

13 

1  2  3  4  5  6  7  8  9  10  11  12  13  14  IS  16  17  18  19  20  21  22  23  24  25 

13 

26  27  28  29  38  40  45  47  56  59  60  61  62  63  64  66  67  68  69  70  71  72  73  74  75 
13 

76  77  78  79  80  81  82  83  84  85 


Figure  2-6.  Example  of  a  HEAVY  ATTACK  Aircraft  versus  Target  Exclusion 
File.  Length  for  the  imclassified  data  set  is  48  lines. 

The  Aircraft  versus  Weapon  Exclusion  File  is  similar,  except  the  numbers 
on  the  following  row  of  each  pair  are  the  weapon  types  which  will  never  be 
expended  by  its  corresponding  aircraft  t3pe. 
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The  remaining  exclusion  file  is  the  Aircraft/Target  versiis  Weapon 
Exclusion  File  shown  in  Figure  2-7.  Here,  the  first  row  of  each  pair  contains  the 
aircraft  type  followed  by  the  target  type.  The  following  row  is  the  weapon  types  which 
wiU  never  be  e^qiended  by  that  aircraft  against  that  target. 


1.  5 

2  *  6  8  9  10  11  12  14  15  17  20  23  24  25  34  42  46  53  55 

1.  5 
57  58 
1,12 

2  4  6  8  9  10  11  12  14  IS  17  20  23  24  25  26  27  34  38  46 

1,12 

47  51  53  55  57  58 

• 

• 

99,85 

1  2  3  4  5  6  7  8  9  10  11  12  14  IS  18  19  23  24  42  46 

99,85 

47  55  57  58 

Figure  2-7.  Example  of  a  HEAVY  ATTACK  Aircrafl/Target  versus  Weapon 
Exclusion  File.  Length  for  the  unclassified  data  set  is  244  lines. 


D.  OUTPUT  FROM  HEAVY  ATTACK 


HEAVY  ATTACK  produces  three  reports:  short,  medium,  and  long.  The  Combat 
Forces  Division  analyst  mainly  uses  the  short  report  (Figure  2-8).  It  is  divided  into  two 
sections:  residts  for  each  time  period  and  then  cumulative  results  through  all  seven 
periods.  In  the  first  section,  data  concerning  weapons  expended  by  aircraft  category 
and  target  types  destroyed  are  listed  for  each  time  period.  The  second  contains  the 
same  data  as  the  first,  accumulated  through  the  time  periods,  plus  the  distribution  of 
weapon  types  to  target  types.  The  final  data  table  listed  in  the  report  is  the 
cumulative  and  maximum  allowable  expenditure  of  weapons  used  during  the  "war*. 

The  medivun  report  is  also  divided  into  two  sections:  results  for  each  time  period 
and  cumulative  results.  The  first  section  lists  the  expected  kills  of  targets  by  aircrafi:, 
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WEAPON 

TOTAL 

END  OF  FERICH)  1 

TYPE 

EXPENDED 

-CAT  A- 

-  -CAT  B— 

-CAT  C - CAT  D— 

-CAT  E- 

-  -CAT  F— 

-CAT  G— 

WPOl 

187* 

1316 

0 

0 

0 

0 

0 

0 

WP03 

362 

362 

0 

0 

0 

0 

0 

0 

WPO* 

5*02 

1871 

0 

923 

3*5 

0 

2162 

99 

WP07 

882 

0 

882 

0 

0 

0 

0 

0 

• 

• 

• 

• 

• 

• 

WP38 

16* 

0 

0 

0 

0 

110 

0 

53 

TARGET 

RECONSTITUTION  AND  RESUPPLY 

END  OF  PERIOD 

1 

TARGET 

STARTING 

PRIOR 

NEW 

FRAC  TARGETS 

CUM 

NEWLY 

CUM 

RESULT 

TYPE 

TARGETS 

TARGETS 

TARGETS 

FIXED  FIXED 

FIXED 

KILLED 

KILLED 

TARGETS 

1  TGOl 

16. 

16. 

0. 

0.*S1 

0. 

0. 

16. 

16. 

0. 

2  T602 

8. 

8. 

0. 

0.777 

0. 

0. 

0. 

0. 

8. 

3  TG03 

32*. 

32*. 

0. 

0.777 

0. 

0. 

0. 

0. 

32*. 

*  TGO* 

162. 

162. 

0. 

0.777 

0. 

0. 

0. 

0. 

162. 

• 

• 

• 

• 

• 

85  T685 

0. 

0. 

0. 

1.000 

0. 

0. 

0. 

0. 

0. 

TARGET 

TOTAL  TARGETS 

PERCENT 

SORTIES 

KILLED 

KILLED 

POT  KILL 

POT  VALUE 

TGOl 

*0 

16.00 

100 

16.0 

22*.  0 

TG08 

138 

10.36 

*9 

10.9 

272.6 

T616 

206 

20.65 

9* 

22.0 

550.0 

TG18 

221 

*2.00 

100 

*2.0 

8*0.0 

• 

♦ 

• 

• 

TG79 

21 

2*.  00 

100 

2*.0 

360.0 

TARGET/WEAPON  SIMMRY  fuR 

TIME  PERIOD (S)  1-  1 

TARGET  WPOl  WP02 

WP03  WPO*  WP05 

WP06  WP07 

WF08 

WP09  WPIO 

WPll 

WP12 

TG08 

556  0 

0 

0  0 

0  0 

0 

0 

0 

0 

0 

T618 

0  0 

0 

0  0 

0  882 

0 

0 

0 

0 

0 

T619 

0  0 

0 

0  0 

0  0 

0 

0 

*19 

980 

0 

TG22 

0  0 

0  *527  0 

0  0 

0 

0 

0 

0 

0 

• 

• 

• 

• 

• 

• 

TG79 

56  0 

0 

0  0 

0  0 

0 

0 

0 

0 

0 

WEAKW 

TYPE 

HPOl 

WF02 

WF03 

WPO* 

• 

HF38 


CUMULATIVE 

EXPENDITURE 

7710 

0 

4128« 

23551 


MAXIMUM 

EXPENDITURE 

7710 

5*7 

*128* 

9999999 

270 


Figure  2-8.  Example  of  a  HEAVY  ATTACK  Short  Report.  Length  for  the 
unclassified  data  set  is  1500  lines. 


the  allocation  of  aircraft  sorties  to  targets,  and  the  expenditure  of  weapons  by  aircraft. 
The  second  part  accumulates  the  same  aircraft  allocation  and  weapons  expenditure 
data  (see  Figure  2-9). 
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WEATHER  DISIRIBUTIOHS 

1  0.05 

1  0.00 

2 

2 

0.20 

0.21 

3  0.22  4 
3  0.23  4 

0.14 

0.15 

5  0 
5  0 

08  6 
08  6 

0.31 

0.33 

******  AGGREGATE  EXPECTED  KILLS  PER  SORTIE  ***' 

k** 

ACOl 

AC02 

AC03 

AC04 

AC05 

AC06 

AC07 

AC08 

AC09 

ACIO 

AC13 

1  TGOl 

0.127 

0.193 

0.000 

0.242 

0.442 

0.242 

0.000 

0.000 

0.144 

0.302 

0.127 

2  TG02 

0.000 

0.000 

0.000 

0.000 

0.175 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

3  TG03 

0.000 

0.118 

0.000 

0.000 

0.518 

0.000 

0.169 

0.169 

0.000 

0.000 

0.000 

4  T604 

0.000 

0.000 

0.000 

0.000 

0.408 

0.000 

0.000 

0.000 

0.000 

0.000 

»•  0.000 

• 

• 

• 

• 

• 

• 

85  T6B5 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

ALLOCATION  OF  SORTIES  TO  TARGETS 

AIRCRAFT  TYPES 

TGT-(VAL) 

ACOl 

AC02 

AC03 

AC04 

AC05 

AC06 

AC07 

AC08 

AC09 

ACIO 

AC13 

817 

305 

189 

169 

150 

380 

356 

330 

0 

138  •< 

211 

TGOl (14. 00) 

0 

0 

0 

0 

40 

0 

0 

0 

0 

0 

0 

TG02(10.00) 

TG03(  1.00) 

TG04(  1.00) 

110 

0 

21 

0 

0 

0 

0 

0 

0 

0 

0 

• 

• 

• 

• 

• 

•4 

TGSSdO.OO) 

ACFT  SORTIES  817 

305 

189 

169 

150 

380 

356 

330 

0 

138 

211 

TGTS  KILLED 

267 

51 

34 

68 

74 

68 

23 

53 

0 

10 

52 

EXPENDITURE 

PER  SORTIE 

WPH  TYPES 

AIRCRAFT  TYPES 

ACOl 

AC02 

AC03 

AC04 

AC05 

AC06 

AC07 

AC08 

AC09 

ACIO 

AC13 

WPOl 

1.444 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

3.557 

0.149 

WP03 

0.400 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.029 

WP04 

0.793 

0.000 

4.501 

1.875 

0.000 

5.004 

0.001 

0.249 

0.000 

0.000  ••  5.258 

WP07 

0.000 

2.665 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

• 

• 

• 

• 

• 

• 

WP38 

0.000 

0.000 

0.000 

0.000 

0.694 

0.000 

0.003 

0.131 

0.000 

0.000 

0.000 

Figure  2-9.  Example  of  a  HEAVY  ATTACK  Mediiun  Report.  Length  for  the 
xmclassifled  data  set  is  2700  lines. 


The  long  report,  typically  over  25,000  lines,  contains  all  the  information  of  the 
short  and  medium  reports,  plus  additional  information  produced  by  the  optimizer. 
Some  additional  information  is  evaluated  by  the  analyst  and  the  theater  team,  but  the 
m^ority  of  the  information  is  useful  only  in  diagnosing  any  data  errors  identified  by 
the  optimization. 
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m.  HEAVY  ATTACK  DECISION-MAKING  PROCESS 


Analjrsis  of  HEAVY  ATTACK  results  is  a  tedious  process.  HEAVY  ATTACK 
generates  a  tremendous  amount  of  output.  Because  there  is  only  one  week  for  the 
planning  conference  the  theater  team  scarcely  has  time  for  a  thorough  and  in-depth 
analysis.  In  addition,  multiple  runs  of  HEAVY  ATTACK  are  required  in  order  to 
generate  acceptable  results,  further  exacerbating  the  problem. 

A.  STEPS  TOWARD  A  SOLUTION 

Three  steps  are  employed  —  analyze  the  decision-making  process,  design  analysis 
tools,  and  demonstrate  them.  The  first  step  examines  how  the  theater  teams  and 
ciurent  Combat  Forces  Division  analyst  and  operator  of  HEAVY  ATTACK,  MAJ  Lance 
Buckingham,  USAF,  make  their  decisions.  Analysis  choke  points  are  identified. 
Second,  a  design  is  proposed  to  improve  the  time  and  quality  of  analysis.  Finally,  this 
design  proposal  is  demonstrated  with  prototypic  computer-based  tools.  A  scenario  will 
serve  to  illustrate  the  design. 

B.  THE  DECISION-MAKING  PROCESS 

Figure  3-1  shows  the  HEAVY  ATTACK  decision-making  process.  When  a  theater 
team  arrives  at  the  Pentagon  for  their  planning  conference,  the  first  step  in  the 
decision-making  process  is  to  create  the  HEAVY  ATTACK  input  files.  These  files  are 
manually  reviewed  for  correctness.  The  Best  Munitions  Table  Generator  program  is 
used  to  assist  in  reviewing  the  exclusion  files.  Errors  in  the  exclusion  files  are 
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Figure  3-2.  NCAA  Decision-Making  Process 


corrected,  and  the  Generator  program  is  re-nm.  This  cycle  continues  until  the  files 
appear  error-free. 

All  input  files  are  then  passed  to  HEAVY  ATTACK,  and  the  model  is  run.  The 
Combat  Forces  Division  analyst  evaluates  the  output  to  validate  the  model.  The 
analyst  will  then  present  the  model’s  results  to  the  team,  who  will  decide  if  any 
adjustments  need  to  be  made  to  the  input  files.  The  files  are  updated  and  the  model 
is  re-run.  This  cycle  continues  until  the  team  achieves  their  final  results.  The  team 
may  also,  diuing  this  cycle,  send  the  updated  exclusion  files  back  into  the  Table 
Generator  program  to  re-validate  the  tables.  When  the  team  is  satisfied  with  the 
results  from  HEIAVY  ATTACK,  their  weapon  requirements  are  combined  with 
requirements  from  the  other  theaters  an  an  input  into  HEAVY  GOAL. 
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Two  tn^or  analysis  problems,  or  choke  points,  impede  decision-making.  First  is 
the  Table  Generator  program,  which  takes  significant  time  to  run  and  to  evaluate. 
The  second  is  the  actual  analysis  of  HEAVY  ATTACK  Each  of  these  choke  points  is 
amenable  to  automated  analysis  tools. 


i 
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IV.  ANALYSIS  TOOLS  FOR  HEAVY  ATTACK 


In  order  to  see  how  to  improve  the  analysis  of  HEAVY  ATTACK,  one  must 
understand  how  the  analyst /theater  teams  presently  use  the  model.  Figure  4-1  is  a 
current  HEIAVY  ATTACK  Aircraft  Data  File.  There  is  an  obviovts  problem:  this  file 


SEL#  SAB#  ATT# 

va# 

TEST 

SPECIAl. 

CAPABILITY 

CMC 

ARC 

110 

2 

ACU 

0 

0 

2  2  0 

2 

AC2 

0 

0 

3  S  0 

1 

ACSA 

0 

0 

• 

• 

• 

• 

• 

• 

13  1  0 

2 

ACIB 

0 

0 

.OS  .20  .22 

.1* 

.08  .31 

—ACl . AC2-- 

-  --AC5A—  - 

■ACSB - AC8— 

— AC6- 

—  AC3A/B/C 

■AC3D/E - AC*— 

A/C#  AIRCRAFT 

PERI 

PER2 

FER3 

FER4 

PS15 

FER6 

FER7 

1  ACIA 

688 

1221 

1008 

17*7 

1366 

1272 

0 

1 

0.82 

0.92 

0.92 

0.92 

0.92 

0.92 

0.00 

2  AC2 

331 

391 

309 

510 

360 

337 

0 

2 

0.92 

0.92 

0.92 

0.92 

0.92 

0.92 

0.00 

3  ACSA 

20S 

2SS 

368 

630 

525 

516 

0 

3 

0.92 

0.92 

0.92 

0.92 

0.92 

0.92 

0.00 

• 

• 

• 

# 

• 

13  ACIB 

222 

305 

2S2 

*37 

3*2 

318 

0 

1 

0.9S 

0.95 

0.95 

0.95 

0.95 

0.95 

0.00 

Figure  4-1.  HEAVY  ATTACK  Aircraft  Data  File. 


contains  a  lot  of  numbers  with  veiy  little  associated  context.  Because  HEAVY 
ATTACK  is  a  FORTRAN  program,  its  output  may  be  influenced  by  a  single  mis  aligned 
input  field.  The  Target  and  Weapon  Data  Files  (Figures  2-4  and  2-5)  also  exhibit  this 
weakness. 

The  exclusion  files  also  present  nugor  problems.  Figure  4-2  is  a  restatement  of 
the  Aircraft/Target  versus  Weapon  Exclusion  File.  As  represented  by  the  figure,  these 
files  are  all  numbers  with  no  associated  context.  For  example,  the  first  and  second 


lines  of  Figxire  4-2  exclude  aircraft  type  1  fi-om  attacking  target  type  5  with  weapon 
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1,  5 

2  4  6  8  9  10  11  12  14  IS  17  20  23  24  2S  34  42  46  S3  SS 

1.  5 
'7  S8 
.,12 

2  4  6  6  9  10  11  12  14  IS  17  70  23  24  2S  26  27  34  38  46 

1,12 

47  SI  S3  SS  S7  S8 

• 

• 

99, 8S 

1  2  3  4  S  6  7  8  9  10  11  12  14  IS  18  19  23  24  42  46 

99, 8S 

47  SS  S7  S8 


Figure  4-2.  Example  of  an  imclassified  HEAVY  ATTACK  Aircrafl/Target 
versus  Weapon  Exclusion  File. 

types  2,  4,  6,  and  so  on.  The  exclusion  files  are  the  nugor  soiu*ce  of  errors  with  the 
input.  Because  of  the  difficulty  in  identifying  errors  in  these  files,  the  Table 
Generator  program  was  created. 

After  HEAVY  ATTACK  is  nm,  the  output  reports  are  generated.  Figure  2-8  is 
just  a  portion  of  an  unclassified  Short  Report,  which  contains  information  for  each  of 
the  seven  time  periods  plus  accumulation  across  the  seven  periods.  The  week-long 
planning  conference  provides  little  time  to  thoroughly  analyze  the  HEIAVY  ATTACK 
reports. 


A.  ANALYSIS  IMPROVEMENTS 

The  next  step  was  to  design  a  computer  ^tem  which  might  provide  analysis 
improvements.  Several  commercially  available  programs  were  reviewed  and  Borland 
International’s  Quattro  Pro  3.0  spreadsheet  program  was  chosen.  Quattro  Pro  provides 
superior  graphic  capabilities  [Ref  9,10,11,12]  and  "prints”  to  text  (ASCII)  files.  This 
text  file  capability  is  extremely  important  in  order  to  transfer  data  back  and  forth 
between  desktop  computers  (Quattro  Pro)  and  mainframes  (HEAVY  ATTACK). 
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Further,  Quattro  Pro  provides  the  ability  to  import  a  text  file  and  parse  the 
information  into  usahle  spreadsheet  cells. 

The  Quattro  Pro-based  analysis  tool  has  been  named  QuikDraw.  It  is  divided 
into  two  sections;  Data  Entry  and  Output  Listing. 

1.  Data  Entry 

The  Data  Entry  handles  the  data  inputs  for  HEAVY  ATTACK  — 
specifically,  the  data  and  exclusion  files  described  in  Chapter  n.  The  remaining  "other" 
files  are  updated  by  the  Combat  Forces  Division  analyst  using  the  mainframe 
computer’s  text  editor.  The  HEAVY  ATTACK  Target  and  Weapon  Data  Files  are 
combined  into  a  single  Quattro  Pro  worksheet.  The  Aircraft  Data  File  has  its  own 
Quattro  Pro  worksheet  (Figure  4-3).  (The  term  worksheet  is  synonymous  herein  with 
file). 

Currently  HEAVY  ATTACK  lists  combinations  of  aircraft,  targets,  and 
weapons  which  are  militarily  infeasible  or  invalid.  However,  the  theater  teams  tend 
to  think  in  terms  of  valid  combinations.  Therefore,  QuikDraw  has,  for  each  HEAVY 
ATTACK  exclusion  file,  a  corresponding  incluaion  worksheet.  Now,  the  users  indicate 
with  X’s  the  valid  combinations  in  the  worksheets.  Figure  4-4  shows  the  unclassified 
Aircraft  versus  Target  Inclusion  Worksheet. 

The  Best  Munitions  Table  Generator  program,  which  is  the  first  choke 
point,  is  replaced  by  the  inclusion  worksheets.  Hence,  there  is  no  need  to  execute  the 
Generator  program  and  analyze  the  generated  tables. 
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Figure  4-3.  Example  of  an  unclassified  QuikDraw  Aircraft  Data  Worksheet. 
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Figure  4-4.  Example  of  an  imclassifled  QuikDraw  Aircraft  versus  Target 
Worksheet. 
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After  a  user  makes  changes  to  any  Data  Entiy  worksheet  and  is  ready  to 
re-run  HEAVY  ATTACK,  he  executes  a  Quattro  Pro  macro  program  embedded  in  that 
worksheet.  That  macro  will  write  the  worksheet  data  to  a  text  file.  (For  simplicity, 
the  macro  is  executed  with  the  same  keystroke  in  each  worksheet).  Then,  a 
FORTRAN  program  is  executed  to  convert  the  text  files  into  the  format  required  for 
HEAVY  ATTACK  This  is  necessary  because  HEIAVY  ATTACK  as  a  FORTRAN 
program,  requires  its  input  data  to  be  in  specific  columns. 

2.  Output  Listiiig 

The  Output  Listing  section  has  four  worksheets;  they  are  listed  in  Table 
I.  Within  each  worksheet  are  three  graph  types:  single  run,  multiple  run,  and 
cumulative.  The  single  nm  graphs  display  information  for  the  ciurent  nm;  multiple 
graphs  present  information  from  two  successive  nms.  The  cumulative  graphs  display 
"course  of  the  war"  information.  All  graphs  display  percentages,  with  raw  numbers 
available  below  the  graphs. 

Table  I.  QUIKDRAW  OUTPUT  WORKSHEETS.  THESE  WORKSHEETS  ARE 
CAPABLE  OF  GENERATING  THREE  TYPES  OF  GRAPHS.  EXAMPLES  OF  THESE 
GRAPH  TYPES  ARE  SHOWN  IN  FIGURES  4-5  THROUGH  4-8. 

Aircraft  Attack  Missions  Flown  by  Time  Period 

Weapon  Expended  by  Time  Period 

Weapon  IVpes  Expended  across  Time  Periods 

• 

Target  l^pes  Destroyed  across  Time  Periods 
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There  is  a  fundamental  difference  between  the  *1)3^  time  period"  and  the 
"across  time  period"  worksheets.  For  the  former  the  graphed  values  are  the 
percentages  for  each  time  period.  Thus,  the  graphed  values  stun  to  100  percent.  In 
the  latter  case,  the  graphed  values  are  the  percentages  over  all  time  periods,  and  may 
exceed  100  percent  due  to  target  regeneration. 

A  special  HEAVY  ATTACK  report  (DWUNCL)  was  created  to  export  data 
to  QuikDraw.  This  report  contains  the  data  necessary  to  generate  the  graphs,  plus 
some  specific  syntactic  semaphores  for  the  Quattro  Pro  parsing  function.  The  parsing 
semaphores  allow  Quattro  Pro  to  identify  only  the  necessary  columns  from  a  report 
table,  significantly  reducing  the  amoimt  of  data  kept  in  the  worksheets. 

Figures  4-5  and  4-6,  Attack  Missions  Flown  and  Weapons  Expended,  are 
examples  of  "by  time  period"  graphs.  With  these  graphs  the  analyst  and  the  theater 
team  can  instantly  see  how  the  aircraft  were  allocated,  or  weapons  expended  in  tune 
period  one.  This  analysis  takes  minutes  rather  than  the  tens  of  minutes  or  horns 
required  to  analyze  the  current  reports.  In  addition  this  analysis  is  more  thorough. 
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Figure  4-5.  QuikDraw  ciirrent  run  graph  of  Aircraft  Attack  Missions  Flown  by 
Time  Period  for  time  period  1  tising  the  unclassified  data  set.  Percentages  sum 
to  100. 


Figure  4*6.  QuikDraw  current  run  graph  of  Weapons  Expended  By  Time 
Period  for  time  period  1  using  the  imclassiBed  data  set.  Percentages  sum  to 
100. 
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Another  feature  of  QuikDraw  is  the  ability  to  view  multiple  runs.  Figure 
4-7  is  an  example  of  a  mviltiple  run  graph,  showing  how  the  expenditure  of  weapons 
is  affected  by  removing  an  aircraft  from  the  data  set.  To  produce  this  graph,  HEAVY 
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Figure  4-7.  QuikDraw  multiple  rvm  graph  of  Weapons  Expended  Across  Time 
Periods  for  time  period  1  using  the  unclassified  data  set. 

ATTACK  first  executed  with  thirteen  aircraft  types  in  the  data  set.  The  weapons 
expended  by  those  aircraft  are  represented  by  light-shaded,  double  cross-hatch  bars 
labeled  Last  (run).  Aircraft  AC6,  having  flown  about  thirteen  percent  of  the  missions 

in  the  first  run,  was  removed  from  the  data  set  and  the  model  re-run.  The  weapon 

• 

types  expended  by  the  remaining  twelve  aircraft  are  represented  by  the  heavy-hashed 
bars  labeled  Current  (run).  The  analj^t/theater  teams  are  then  able  to  analyze  how 
removing  an  aircraft  from  HEAVY  ATTACK  affect  the  expenditure  of  weapons.  With 
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QilikDraw  the  analysis  takes  minutes,  rather  than  the  hours  analyzing  the  HEAVY 
ATTACK  reports  by  hand. 

Finally,  QuikDraw  allows  the  analyst/theater  team  to  view  (nimulative 
results  over  "the  course  of  the  war"  (all  seven  time  periods).  An  example  is 
Figure  4-8  showing  how  the  target  data  base  was  destroyed  during  the  "war".  Here  five 
of  38  targets  had  higher  than  100  percent  destuction  because  of  HEAVY  ATTACK’S 
target  regeneration  featme. 

B.  TESTING  AND  IMPLEMENTATION 

QuikDraw  was  tested  using  three  different  subsets  from  an  unclassified  data  set 
provided  by  the  United  States  Air  Force  Combat  Forces  Division.  The  tests  generated 
high  confidence  that  Data  Entiy  worksheets  provide  HEAVY  ATTACK  with  relatively 
quick-entry  and  error-free  input.  The  tests  also  generated  all  three  graph  types  for 
the  four  Output  Listing  worksheets. 

QuikDraw  consists  of  12  files,  24  graphs,  nearly  178,000  spreadsheet  cells,  and 
1,100  lines  of  Quattro  Pro  macro  coding  using  nearly  1,900  macro  instructions.  The 
FORTRAN  code  to  convert  the  QuikDraw  Data  Entry  worksheets  into  the  HEIAVY 
ATTACK  input  files  consists  of  581  lines.  QuikDraw  is  available  fi'om  the  author  or 
Professor  Bradley. 
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Periods  for  time  periods  1—7  using  the  unclassified  data  set.  Total  percentages 


may  exceed  100%  of  available  target  types  due  to  regeneration. 
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V.  CONCLUSION 


HEAVY  ATTACK  is  an  important  operations  research  model.  It  has  been  used 
for  over  20  years  and  may  continue  to  be  used  in  the  foreseeable  future.  The 
improvements  in  analyzing  this  model  offer  significant  savings  in  man-hours  and 
money. 

HEIAVY  ATTACK  is  just  one  example  of  using  a  spreadsheet  as  a  tool  to  analyze 
output.  The  approach  developed  here  to  (1)  analyze  a  decision-making  process,  (2) 
design  analysis  tools,  and  (3)  demonstrate  them,  can  be  applied  to  other  large-scale 
optimization  models. 

This  thesis  designed  and  implemented  a  state-of-the-art  output  analysis  i^stem 
that  demonstrates  analysis  choke  points  can  be  reduced  or  eliminated.  The  current 
cumbersome,  time-constuning  data  input  and  quality  assurance  process  is  replaced  by 
QuikDraw  which  does  the  same  work  in  significantly  less  time,  reduces  errors  in  the 
input  files,  and  minimizes  the  time  spent  analyzing  hundreds  of  sheets  of  output 
through  the  use  of  graphics.  In  total  this  improves  the  quality  of  analysis. 

A  preliminary  version  of  QuikDraw  was  demonstrated  to  the  United  States  Air 
Force  Combat  Forces  Division  in  Washington  D.C.  in  December  1991.  As  a  result  of 
the  demonstration,  they  intend  to  incorporate  QuikDraw  into  the  next  generation 
NCAA  process,  which  will  be  based  on  a  486  personal  computer. 
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